Skip to content

fix: treat a reference with a /, ? or # before its first colon as a relative IRI - #371

Open
jdsika wants to merge 13 commits into
digitalbazaar:masterfrom
jdsika:fix/iri-resolve-slash-before-colon
Open

jdsika wants to merge 13 commits into
digitalbazaar:masterfrom
jdsika:fix/iri-resolve-slash-before-colon

Conversation

@jdsika

@jdsika jdsika commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Rebased on master now that #372, which this fix needs (see below), is merged. The diff is this PR's own 13 commits.

Summary

iri_resolver.resolve() treats any reference that contains a : as an absolute IRI. A relative reference such as /a:b, x/y:z or a?b:c is therefore returned unresolved, and jsonld.to_rdf() then drops the triple without an error.

from pyld import jsonld
from pyld.iri_resolver import resolve

print(resolve("/a:b", "http://example.org/ns/doc"))
# master:  /a:b
# this PR: http://example.org/a:b

doc = {"@id": "/a:b", "http://example.org/p": "x"}
print(jsonld.to_rdf(doc, {"format": "application/n-quads", "base": "http://example.org/ns/doc"}))
# master:  (empty: the triple is dropped)
# this PR: <http://example.org/a:b> <http://example.org/p> "x"^^<http://www.w3.org/2001/XMLSchema#string>  .

Cause: resolve() decides that a reference has a scheme with relative_iri.find(":") >= 0.

Fix: a scheme cannot contain a /, ? or # (RFC 3986 §3.1 and Appendix B), so a reference with one of them before its first : is relative.

Details

Tests

tests/iri_resolver/test_resolve.py gets fourteen cases, all of which fail on master: the three tests from rubensworks/relative-to-absolute-iri.js#28; /a:b against a base; a?b:c and a#b:c; //h/./a:b; dot segments in a query or fragment; /a:b/../c against http://base.org and x/./y:z against tag:example, as in the JS PRs; and //h/a:b/.?q against http://x/ and z/a:b/.? against urn:x, which the previous version of this PR resolved without the / of the final dot segment. The full suite passes: 1965 passed, 27 skipped, 1 xfailed.

resolve() was compared with a strict RFC 3986 §5.2 resolver over 782,233 reference/base pairs: a hand-picked set of 4,730, the same references with /.?q, /..?q, /.#f and /..#f appended (13,588), 100,000 random pairs, and 663,915 pairs built from every path of up to three segments taken from '', ., .., a, .a and a:b, with or without a scheme or host, and with or without a query or fragment. Nothing that master, #372 or the previous version of this PR resolves correctly changes, and PyLD now gives the same results as relative-to-absolute-iri.js with rubensworks/relative-to-absolute-iri.js#33 and rubensworks/relative-to-absolute-iri.js#34 on all of them. An earlier comparison over the first 4,730 pairs missed the regression above, because none of those references had a segment followed by a final . or .. and a query or fragment.

Found while cross-checking RDFLib's JSON-LD output with PyLD.

@mielvds

mielvds commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Thanks @jdsika, this looks good. I'm going to hold on until there's a response on rubensworks/relative-to-absolute-iri.js#34 to prevent us drifting.

@jdsika jdsika changed the title fix: treat a reference with a slash before its first colon as a relative IRI fix: treat a reference with a /, ? or # before its first colon as a relative IRI Nov 12, 2026
@jdsika

jdsika commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

rubensworks found a regression in the JS twin of this PR (rubensworks/relative-to-absolute-iri.js#34), and this PR had it too: once a : after a / no longer counts as the end of a scheme, remove_dot_segments() gets the whole path and drops the / of a final dot segment before a query, so //h/a:b/.?q against http://x/ gave http://h/a:b?q. My comparison with an RFC resolver had no case like that, so I missed it.

The fix is in #372, so this branch now merges it (de9c1fb), the same way the JS PR builds on its twin. 33cb102 adds the two cases as tests, 6b1c835 ports two more tests from the JS PRs, and fdde386 is a small cleanup that changes no result. Until #372 is merged, the diff here includes it. With both, PyLD gives the same results as the JS PRs on all 782,233 reference/base pairs I compared.

@jdsika
jdsika force-pushed the fix/iri-resolve-slash-before-colon branch from daa6bdb to 2142afb Compare November 13, 2026 08:52
@mielvds

mielvds commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

@jdsika could you rebase this on master?

…ive IRI

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…ative

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…gments

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…elog entry

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
'//h/a:b/.?q' against 'http://x/' and 'z/a:b/.?' against 'urn:x' lost
the '/' of the final dot segment on this branch before it built on
pyld#372, the same regression that relative-to-absolute-iri.js#34 had.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
After the scheme check, value_colon_pos was always -1, so pass -1 to
remove_dot_segments_of_path() directly. Find the start of the path
with one find() and a check for '?' or '#' before it, and document
colon_position as the ':' that ends the scheme, or -1. This changes no
result. It matches relative-to-absolute-iri.js#34.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…s does

Port the two cases of relative-to-absolute-iri.js#33 and digitalbazaar#34 that had
no twin here: '/a:b/../c' against 'http://base.org' and 'x/./y:z'
against 'tag:example'.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Without a base IRI, such a reference is still returned unchanged, as
in relative-to-absolute-iri.js.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
@jdsika
jdsika force-pushed the fix/iri-resolve-slash-before-colon branch from 2142afb to c010df3 Compare November 19, 2026 10:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants